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ABSTRACT 



This paper aims to situate a research project in the context 
of the literature about writing, describe the goals and methods of the 
project, and present some empirical results from the initial stages of the 
project. Students in four schools were divided into an experimental group 
(N=43 ) and a control group (N=48) . The experimental group received specially 
designed writing tasks and the control group did the homework orally. Four 
types of writing tasks were used: (1) description of experimental 

observations; (2) interpretation of experimental observations; (3) 
explanation of phenomena; and (4) planning a clarifying experiment in the 
face of a remaining problem. Students’ assessments are analyzed for the 
appropriate use of chemical facts, topic relevance, and text coherence. It 
was concluded that the appropriate use of chemical facts and concepts 
remained low throughout the study and that there were no significant 
differences between the two groups. Contains 26 references. (DDR) 
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Objectives 



The aim of the paper is to situate our project in the context of the research on 
writing and to describe the aims and methods of the project. Additionally, we 
present some empirical results from the initial stages of the project. 



Theoretical framework 

Language seems to be central for communicating and more importently for 
understanding scientific phenomena (Stork 1993). Students learn through the use 
of language by: describing daily life phenomena, finding out the differences and 
similarities of these phenomena, creating terms for the similarities and deriving 
rules, and explaining phenomena with the help of abstract theories and concepts. 
This could be a plea for systematic class discussion where students thoroughly 
discuss their ideas and reach common conclusions. But class discussions have 
shortcomings like all spontaneous oral comments made under the influence of 
relevant events. Speaking is "full of repetition, unclear (...) (and) also often 
formally incorrect" (Aebli 1991, p. 157). In contrast, writing is an instrument to 
complete, to extend and to permanently consolidate the process of knowledge 
and thinking. 

Cognitive science and its emphasis on "epistemic writing" (cf. Bereiter 1980) 
supports Aeblis' pedagogical common sense. Epistemic writing is a form of 
processing one's own knowledge. This knowledge is not only a precondition for 
producing texts, but text production causes the writer to think over his/her ideas 
once again, bringing them into new context, giving them a new structure, and 
finally arriving at newly conceived knowledge. In the context of this research 
strand on writing, our project can be classified as a contribution to the role and 
impact of writing in cognitive functioning (cf. e.g. Bereiter and Scardamalia 1978, 
Britton 1975, Bruner 1972, Olson 1977, Vygotsky 1978). 

Does writing help students to learn science more meaningfully than traditional 
class discussion? If it does, how? If it doesn't, why not? The aim of the project is 
to test whether writing results in more meaningful science learning (Ausubel, 
Novak and Hanesian 1980), and better long-term recall. More than that, the 
analysis includes a look into the "black box" to consider the exact ways in which 
students' writing changes. 
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Data and Methods 

The writing project was carried out in a 9th grade chemistry class at each of four 
different Gymnasiums (secondary schools). Teachers implemented four units 
covering a total of 30 lessons. These teaching units are part of a course 
developed and tested at the Institute for Science Education and used for 
introductory chemistry classes. This course distinguishes itself by offering a 
sequence of experiments which takes the daily-life and environmental 
surroundings into consideration (Stork/Schulz/Johanssen 1993). The first three of 
these units focus on phenomena which are described and explained on a 
descriptive-operative level. The final units show examples of evaporation and 
condensation phenomena where interpretation takes place on the particle level: 

1. A waste product of combustion, which creates problems - carbon dioxide 
and the green house effect. 

2. The most important of all oxides: Water- Circulation of water, provision of 
drinking water, purification of sewage. 

3. A comparitive view of the coversion of energy during chemical reactions - 
the "struggle" of oxygen. 

4. From the level of phenomena to the level of particles - the states of matter 
and their meaning. 

We divided the classes into two groups, an experimental group (with 43 students) 
and a control group (with 48 students). 



Types and assesment of texts. 

The project treatment was as follows. The experimental group received eight 
specially-designed writing tasks as homework, two per teaching unit, while the 
control group did the homework orally. The written products from the experimental 
group were collected and returned to students with comments on grammar and on 
their expression of chemistry concepts. This iterative procedure allowed students 
to use writing to explain chemical phenomena and to receive feedback on their 
writing and understanding of chemistry concepts. 
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Students were asked to complete four different types of writing tasks. Each 
successive type represents an increase in intellectual demands. 

1 . Description of experimental observations including relevant 
circumstances of natural and technical phenomena. 

2. Interpretation of experimental observations based on pre-theoretical, 
concrete observed regularities. 

3. Explanation (interpretation) of phenomena with the help of abstract 
theories and concepts (concept of energy, the particle model). 

4. Planning a clarifying experiment in the face of the remaining problem. 

The written texts were analyzed for 1) the appropriate use of chemical facts and 
concepts. We developed models listing elements of an appropriate answer for 
each task. Students' answers were then compared to these models. The written 
texts were also analyzed for 2) topic relevance (i.e., adhering to the task; 
McCutchen 1 986), and 3) "text coherence" (van Dijk/Kintsch 1 983) or the logical 
structure of the text. 

'Text coherence" includes two different categories, the "logical structure of the 
text" and "cohesion connections". The analysis of the "logical structure of the text" 
works with the assumption that every main clause correlates with the previous 
main clause. In the text, these structural correlations are realized through different 
content correlations or "functional sentence roles" (Cooper/Matsuhashi 1983; 
Eigler et al 1990). We classify a main clause as "superordinate" if in this main 
clause the content of a previous main clause is summarized or generalized. A 
main clause is considered "subordinate" to a previous main clause if the content 
of the previous main clause is explained by using examples, definitions or 
characteristics. We assess a main clause as "coordinate" to the previous clause if 
new information is added on the same level of abstract or if a contrast is 
formulated. With this analysis, we are able to make statements on the linking-up 
of the text and on high complexity of thoughts or on the linearity of the text 
showing a train of thoughts. 

The category "cohesion" is defined "as the set of possibilities that exist in the 
language for making the text hang together" (Halliday/Hasan 1976, p. 18). We 
differentiate between the following cohesions: Pronouns, adverbs, comparatives, 
conjunctions, lexemes and phrases. The amount of the most used cohesions 
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points out the structure of the text and the complexity of knowledge (McCutchen 
1986). 



Types and assesment of tests. 

The following test design was developed in order to find out whether 
independently written observations and interpretations of chemical facts lead the 
students of the experimental group to meaningful learning and whether a long- 
term consolidation of the lesson materials takes place. 





Pre- 

Test 


Interim 
Test 1 


Interim 
Test 2 


Interim 
Test 3 


Interim 
Test 4 


Post- 

Test 


Final 

Test 


Understanding of 
language 


X 










X 




Thinking tied to 
visual conceptions 


X 










X 




Knowledge of 
Chemistry 


X 


X 


X 


X 


X 


X 


X 



Table 1: Types of tests and their contents 



The tests were designed to assess both meaningful learning and long-term 
consolidation of the lesson materials. The study started with a pretest which 
focused on the general ability of language and on the current knowledge of 
chemistry. To assess the students' language faculties and the ability to think in 
visual terms we took parts of the Cognitive Abilities Test (CAT) (Thomdike/Hagen 
1971 ) which was translated and modified for Germany. 

At the end of every teaching unit all students took a test (interim test). Directly 
after the last test of the final unit the students wrote a post-test which covered all 
four units including a second language test. At the end of the school year - two 
months after the post-test - the students took a final exam in order to test their 
retention of the chemistry lessons given during the whole school year. It is 
important to note that these tests were in traditional short answer format. The 
tests required, on the one hand, the recall of learned knowledge and on the other 
hand, the synthesis of knowledge: the reorganization of knowledge and 
application of learned knowledge to new situations or problems. 



All tests were graded for their chemical appropriateness. Together with a prime 
example of the solution the tests were then given back to the students. 
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For our investigation, the tests were analysed under professional points of view. 
As we did for the texts we developed models listing elements of an appropriate 
answer for each task of the tests. Students' answers were then compared to these 
models. 

With these tests we are able to check the following hypothesized effects within the 
experimental group: 

1. During the whole measuring period the amount of the recall of learned 
knowledge should increase and 

2. Writing should lead to a deeper understanding of chemical connections 
which make it easier to reorganize available knowledge and to apply this 
knowledge to new situations or problems. 



Results 1 

The analysis of the texts from the experimental group shows that the appropriate 
use of chemical facts and concepts was relative low throughout the study. In five 
of the eight tasks students' answers contained on average only 37% of elements 
of the model answers. In two of the tasks they scored 50% and in one 70%. 
However, the texts rated highly on topic relevance. The analysis of text coherence 
showed no change in complexity of thoughts over the course of the study. The 
written interpretation of experimental observations based on pre-theoretical, 
concrete observed regularities of as well as the explanation of phenomena with 
the help of abstract theories and concepts (e g., the concept of energy, the 
particle model) remained difficult for the students. Students improved on average 
in their ability to describe experimental observations. For example, in task number 
one the average score was 50% and in task number two it was 70%. There was 
no corresponding increase in students' ability to interpret and explain phenomena. 

Overall, the test results showed no significant differences between the 
experimental and the control group (see diagram 1). However, their were 
important exceptions. 




1 For more detailed results see Nieswandt 1997. 



7 



7 



Sumscore 

Experimental Group versus Control Group 

100 , 

90 

80 

70 



* 60 




e 0,10 0.00 0.12 0.36 0.31 0.12 0.28 

Diagram I 2 



Class 4: Sumscore 

Experimental Group versus Control Group 



60 
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<n 
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o 

E 
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■E-Group 

□C-Group 



Inter 4 Post-test Final test 
0.04 0.11 1.12 



Diagram 2 




2 The differences betweeen the means of each group are often not significant but a look at the 
standard deviation (SD) showed differences between the experimental group and the control 
group as well as between girls and boys. We can say there is a significant difference when the 
difference of SD (this is the difference between mean 1 and mean 2 divide by SD of the whole 
group) is at least .30. We call this a small effect. Differences from .50 are called medium effects 
and differences from .80 are seen as strong effects. This categorization of effects follows Cohen 

(1969, p. 22 ff). With n = 69 scores at s = .30 are significant; with n = 25 scores at s = .50 and 
with n = 10 scores at s = .80 are significant. These statements are valid at p= 0.2, this is similar 
to a power of 0.8. 
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More detailed analysis suggests that in one of the four classes the students of the 
experimental group scored significantly higher in the final test than the students of 

the control group (32% vs. 23%, p < .01 , n = 20; 8 = 1 .12; see diagram 2). 

The analysis of the three different task requirements (recall of knowledge, 
reorganization of knowledge and the application of learned knowledge) show that 
the experimental group of class 4 had a higher ability of recall of chemical 
knowledge at the end of the school year than the students of the control group of 
the same class (see diagram 3). The students of the experimental group also 
have a better long-term recall than they colleagues in the control group. But we 
have not found that the students of the experimental group have had a deeper 
understanding of chemistry. The tests' results of the experimental group were no 
higher than the control group on the ability to reorganize knowledge nor on the 
application of learned knowledge to new situations or problems. 



Class 4: Recall of Learned Knowledge 



Experimental Group versus Control Group 



_c 

O) 

c 

CB 

0 ) 

E 




Diagram 3 



In particular, the comparison of the two groups of girls in class 4 showed that the 
girls of the experimental group have strikingly higher scores at the end of the 
school year than the girls of the control group (see diagram 4). 

er|c 
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Diagram 4 



Because of the small number of students in class 4 we made some more 
qualitative analysis of the students of the experimental group in this class. 

A view of the girls' test results (see table 2) show that only two of the four girls 
have scored higher in most of the interim tests and also in the final test (No. 70 
and 73) than the average score of the whole girls group. One girl also had good 
results in the final test (No. 68) but not in the other tests except for interim test 4. 
In the group of the boys only two boys have scored high in all test (No. 71 and No. 
74). 



No. 


Gender 


Pre- 

test 


Interim 
test 1 


Interim 
test 2 


Interim 
test 3 


Interim 
test 4 


Post-test 


Final test 


68 


female 


21 


26 


17 


24 


40 


29 


37 


70 


female 


40 


47 


36 


44 


29 


29 


43 


72 


female 


6 


50 


28 


39 


47 


30 


25 


73 


female 


13 


29 


42 


52 


44 


34 


37 






65 


male 


15 


41 


26 


56 


37 


27 


- 


66 


male 


13 


38 


13 


29 


24 


25 


25 


69 


male 


17 


29 


11 


29 


18 


29 


29 


71 


male 


40 


59 


61 


51 


36 


44 


35 


74 


male 


23 


50 


33 


52 


52 


44 


35 


75 


male 


30 


50 


31 


33 


23 


41 


28 






All girts 


21.13 


39.41 


28.86 


38.67 


39.94 


30.48 


35.38 


All boys 


22.96 


44.61 


29.17 


41.78 


31.61 


34.93 


30.38 




Table 2: Sumscore of the girts and boys of the experimental group in class 4 
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A look at the single tasks of the tests (see table 3) indicates that the students No. 
70, 71, 73 and 74 have high results in the tasks which required recall of learned 
knowledge and that the students No. 71, 73 and 74 also have high results in the 
tasks which required the synthesis of knowledge: the reorganization of knowledge 
and application of learned knowledge to new situations or problems. 





E- 
Gr. 
Cl. 4 


All 
girls 
Cl. 4 


All 
boys 
Cl. 4 


No. 

70 

f 


No. 

71 

m 


No. 

73 

f 


No. 

74 

m 


Pre-test 


22 


21 


23 




40 






Interim test 1 
Recall 


43 


43 


42 


50 


57 


32 


43 


Reorganization 


35 


20 


47 


20 


60 


40 


80 


Application 


73 


40 


100 


100 


100 


0 


100 


Sumscore 


42 


39 


45 


47 


59 


29 


50 


Interim test 2 
Recall 


21 


18 


23 


29 


63 


23 


6 


Reorganization 


26 


23 


29 


31 


69 


53 


47 


Application 


83 


98 


69 


100 


83 


100 


100 


Sumscore 


29 


29 


29 


36 


61 


42 


33 


Interim test 3 
Recall 


35 


43 


28 


77 


54 


46 


69 


Reorganization 


38 


37 


39 


29 


46 


54 


29 


Application 


44 


38 


49 


44 


53 


53 


65 


Sumscore 


40 


39 


42 


44 


51 


52 


52 


Interim test 4 

Recall 


56 


79 


40 


71 


71 


86 


64 


Reorganization 


16 


19 


14 


8 


8 


23 


30 


Application 


71 


68 


73 


55 


85 


70 


100 


Sumscore 


35 


40 


32 


29 


36 


44 


52 


Post -test 


33 


30 


35 


29 


44 


34 


44 


Final test 


33 


35 


30 


43 


35 


37 


35 



Table 3: Means of the test results of all tests 



Legend: E-Gr.: Experimental group Cl.: Class 

f: female m: male 




Are these results an effect of writing? 

In order to find an answer of this question we looked at first at the results of the 
appropriate use of chemical facts and concepts in all eight written tasks. The 
written tasks belong to different typs of texts: Description of experimental 
observations, interpretation of experimental observations, explanation of 
phenomena with the help of abstract theories and concepts (concept of energy 
and the particle modle), and planning a clarifying experiment. As you see in table 
4, a single written task can include more than one of these types of texts. For 
example the written task 1 includes the description and the interpretation of an 
experimental observation. 



II 
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E-group 

(means) 


Number 


70 

girl 


71 

boy 


73 

girl 


74 

boy 


WT 1 

Description 


4.26 


4 


4 


4 


4 


Interpretation 


3.37 


6 


7 


4 


3 


Sumscore 


7.63 


10 


11 


8 


7 


WT 2 

Description 


9.00 


7 


8 


4 


9 


Sketch 


7.86 


9 


9 


6 


6 


Sumscore 


16.85 


16 


17 


10 


15 


WT 3 

Interpretation 


6.96 


8 


9 


10 


12 


WT 4 

Interpretation 1 


1.52 


1 


3 


1 


1 


Interpretation 2 


4.59 


4 


5 


7 


5 


WT 5 

Explanation 


18.22 


6 


15 


20 


17 


WT 6 

Planing 


3.22 


3 


6 


6 


3 


Sketch 


6.63 


3 


5 


6 


4 


Carrying out 


5.26 


2 


8 


2 


3 


Expectation 


4.93 


2 


12 


3 


2 


Reason 


5.11 


0 


3 


6 


6 


Sumscore 


25.15 


10 


34 


23 


18 


WT 7 

Explanation 1 


7.30 


3 


11 


11 


4 


Explanation 2 


5.11 


3 


7 


7 


2 


WT 8 

Description 


9.93 


9 


8 


10 


0 


Interpretation 


8.89 


15 


9 


14 


0 


Explanation 


5.37 


6 


11 


7 


0 


Sumscore 


24.19 


30 


28 


31 


0 



Table 4: Appropriate use of chemical facts and concepts 

Legend: WT: Writing task 

E-group: Experimental group 



What kind of results did the four selected students have in each of these four text 
types? 

None of the four students have higher than the average scores of the 
experimental group in describing experimental observations. The results in the 
written task 1 and 8 are like the average of the experimental group (see table 4) 
and in the written task 2 it is worse than the experimental group. 



In the second typ of text, the interpretation of experimental observations, all four 
students have scored higher in the written task 3 than the experimental group. 

These results are similar in the written task 1 except for the student No. 74 who 

o 
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has an average result in this written task. The results of the written task 4 show 
that the girl No. 70 scores lower than the average results in both parts of this 
written task while the students No. 73 and 74 score lower only in the first part. In 
the other part they are better. The boy No. 71 has better results in both parts. 

Students had to explain phenomena with the help of abstract theories and 
concepts in the written task 5, 7 and 8. Only the girl No. 73 did this very well in all 
of these three written tasks. The student No 71 who had good results in the first 
two types of tests has lower scores only in the written task No. 5. In the other 
written tasks he scored higher than the average of the experimental group. The 
girl No. 70 has only good results in the last written task and the boy No. 74 scores 
low on all written tasks. 

The good results of the boy No. 71 were also seen in the last text type, the 
planning of a clarifying experiment. He is the only one who had better results than 
the experimental group. The other three students score lower than the average of 
the experimental group. 

Cluster analysis of the variables of the written tasks which give information of the 
topic relevance and the text coherence show that the boy No. 71 was in six written 
tasks in the highest cluster. In the other two written tasks he was in the middle 
cluster. These cluster analysis reveals that the other students are either in the 
middle or in the lower cluster. Only once or twice they are in the highest cluster. 



The results of the analysis of tests and texts show the differences between the 
four students. While the girl No. 70 has good results in the appropriate use of 
chemical facts and concepts only in these written tasks which required the 
interpretation of experimental observations, the male student No. 71 has scored 
high in all types of texts except of the description of experimental observations. 
Because the analysis of topic relevance and text coherence indicates only an 
average writing ability, it is difficult to explain the good test results of the girl No. 
70 as an effect of writing. We find similar results for the students No. 73 and 74. 
But all of them scored high in the final test. Therefore we may say for these three 
students at least, writing results in a better long term-recall. 





We find much better and clearer effects by analysing the boy's No. 71 tests and 
texts. His good appropriate use of chemical facts and concepts and his 
appropriate use of language are also been seen in his good test results. At the 
end of the school year, this boy has a deeper understanding of the chemical 
connections as well as good long-term memory capability. His results indicates at 
least that writing about chemical facts supports a long-term consolidation. 



Summary and final reflections 

The analysis of the tests of the all students who participated in this study does not 
indicate that the students of the experimental group have higher test results than 
the students of the control group except of one class. The students of the 
experimental group in class 4 scored significantly higher in the final test than the 
students of the control group. This result indicates that writing may support a long- 
term consolidation of the lessons materials. 

What are the reasons of these results? 

The analysis of the written tasks have shown that both the appropriate use of 
chemical facts and concepts during the eight written tasks is low, and also the 
language abilities do not meet our expectations. Comments of some students like 
"What, I have to write in chemistry?", "Do we have a chemistry lesson or a 
German lesson?" or "Do not make me crazy, woman!" refer to the low motivation 
of the students to do their written homework carefully. At the same time these 
comments can be interpreted as difficulty experienced during writing. This 
impression is strengthen by the fact that some written homeworks were copied 
from other students or incomplete or the students did them only after repeated 
demands. 

Original writing of a factual text needs at first a thoughtfull structure of the own 
knowledge and then the formulation of a reasonable sequenze of statements. This 
is much more ambitious than the ora! speech. In an oral speech one statement 
can be described and corrected more often by a broaden paraphrasing, new 
examples and proof correction of mistakes. The students should not learn basic 
writing skills in the chemistry lesson; they should apply them and develop further. 
But the majority of students were not able to transfer the abilities and skills of the 
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German class into the school subject chemistry. The high number of spelling, 
grammar and punctuation errors indicate this as well as the low ability of the 
students to write for a special target group. The lag of professional science 
knowledge also makes writing more difficult. Original writing of complex facts - like 
explanation of the three states of matter with the help of the particle model - 
requires that the students have understood the content in their mind thus far and 
that they can elaborate appropriate words and sentences. But when knowledge is 
incomplete (and the results of the analysis of the appropriate use of chemical 
facts and concepts indicate this), then problems already appear during structuring 
the thoughts, which continue in the written products as the language analysis 
makes clear. 

Although the experimental/control group desing of the study made it inpossible to 
practice and teach writing in the class, we did not do completely without it. After 
the students of the experimental group have written their homeworks they 
received a modle of an appropriate answer of this task. In addition they got their 
own writing back after we have corrected mistakes of spelling, grammar and 
punctuation and gave alternatives of phrases. 

The short time periode of the study may be another reason for the modest results 
we see. The entire procedure was finished within half a school year. In face of the 
students' problems with writing this periode of time was not enough. During 
planing this study, we decided not to broaden this periode because we were 
concerned about the variability of the students' motivation. The lose of motivation 
of the few students during the study underscores these concerns. 



Because of the results of the written products should we give up writing of 
chemical facts in chemistry classes? 

The results of class 4 indicate that writing may support the long-term 
consolidation. But the analysis of the four students' individual learning process 
shows clearly only one case where writing about chemical facts was done 
seriously and where both a long-term consolidation, and a deeper understanding 
of chemical connections was achieved. These are preliminary results. More 
analysis of individual students in the other three classes will follow. 

IS 

ERIC 
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The number of succesful cases are small at the moment, but we find some signs 
which suggest there is a positive effect of writing on meaningful! learning. 
Research in science education should note that writing can be an instrument to 
promote science learning. When developing curricula and teaching materials, 
regular written work on scientific contents should be considered in order to 
increase students' abilities and understanding of science. 
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